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SUMMARY 

Commercial  off  the  shelf  (COTS)  products  are  being  used  increasingly  in  military  systems,  an  approach  that  offers 
many  advantages  including  lower  initial  acquisition  costs,  faster  delivery  to  the  front  line  and  ability  to  utilise  the  latest 
advances  in  technology  - a seemingly  perfect  match  to  the  "faster,  better,  cheaper"  ethos  of  modern  acquisition 
initiatives.  COTS  products  do,  however,  bring  their  own  problems,  including  rapid  obsolescence,  lack  of  product 
control  and  fixed  functionality  optimised  for  the  non-military  market.  In  addition  to  addressing  the  complex  technical 
issues  that  the  use  of  COTS  products  brings,  Defence  Ministries  and  Industry  will  have  to  adapt  their  management 
approach  and  practices  if  the  full  potential  of  using  commercial  technology  is  to  be  realised,  and  dangerous  pitfalls 
avoided. 

This  paper  discusses  some  of  the  management  issues  that  will  have  to  be  addressed  and  draws  a number  of  lessons 
relating  to  the  avoidance  of  obsolescence  problems  during  the  in-service  life  of  a system  or  platform. 


BACKGROUND 

The  use  of  commercial  off  the  shelf  (COTS)  components 
is  becoming  an  increasingly  important  aspect  of  the 
acquisition  of  military  systems,  particularly  in  the  areas 
of  information  technology  and  communications.  The  use 
of  COTS  components  should  offer  the  potential  to 
harness  the  rapid  technological  developments  underway 
in  the  commercial  world  and  to  capitalise  on  the  lower 
costs  delivered  by  mass-market  developments.  Over 
recent  years,  these  potential  advantages  have  led  to  a 
view  within  the  defence  authorities  and  in  industry  that 
the  use  of  COTS  was  going  to  solve  many  long  standing 
problems  in  military  systems,  and  would  allow  more 
capable  systems  to  be  delivered  more  quickly,  at  lower 
cost. 

This  initial  widespread  optimism  is  now  being  replaced 
by  a realisation  that  while  the  use  of  COTS  delivers 
many  advantages,  it  also  brings  many  difficulties  and 
challenges  of  its  own.  These  include  more  rapid  product 
obsolescence,  lack  of  control  over  product  support  and 
difficulty  in  predicting  future  developments.  Many  of 
these  difficulties  become  most  critical  after  systems  have 
entered  service  and  the  obsolescence  of  their  components 
has  started  to  have  a significant  effect  on  support  and 
development. 


If  these  problems  with  obsolescence  in  COTS-based 
systems  are  to  be  solved  and  the  attendant  risks 
contained,  then  changes  are  required  to  the  management 
of  their  acquisition  and  support.  Much  work  has  been 
directed  at  the  technical  and  design  issues  relating  to  the 
use  of  COTS  products.  This  paper,  however,  explores  a 
number  of  aspects  of  the  through  life  management  of 
COTS-based  systems,  including  initial  acquisition, 
requirements  management,  managing  upgrades,  spares 
support  and  costing. 

The  paper  focuses  on  information  technology  (IT) 
systems,  as  it  is  in  this  area  that  the  rapid  advances  in 
commercial  technology  produce  the  greatest 
obsolescence  problem.  It  is  hoped,  however,  that  the 
paper  includes  lessons  of  application  in  other  acquisition 
domains. 

This  paper  is  based  on  work  undertaken  by  the  author  for 
the  UK  Ministry  of  Defence  (MOD)  Defence 
Procurement  Agency  (DPA)  and  Defence  Evaluation  and 
Research  Agency  (DERA)  Sea  Systems  Sector  as  part  of 
a series  of  studies  aimed  at  improving  the  COTS 
acquisition  guidance  available  to  UK  MOD  staff.  The 
support  of  all  those  who  contributed  to  these  studies  is 
acknowledged. 


Paper  presented  at  the  RTO  SCI  Symposium  on  ‘‘Strategies  to  Mitigate  Obsolescence  in  Defense  Systems 
Using  Commercial  Components”,  held  in  Budapest,  Hungary,  23-25  October  2000,  and  published  in  RTO  MP-072. 
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COTS  AND  COTS-BASED  SYSTEMS  OVERVIEW 
Terminology 

Before  embarking  on  a discussion  of  COTS  it  is 
necessary  to  define  exactly  what  we  are  talking  about,  as 
common  terms  are  often  used  inconsistently  in  this  field. 
In  this  paper  "COTS"  refers  to  commercial  off  the  shelf 
items,  that  is  those  that  are  developed  for  use  in  the 
commercial  market,  available  from  a catalogue  or  other 
description  and  delivered  fully  developed  and  ready  for 
use.  The  paper  does  not  specifically  address  other  off 
the  shelf  acquisitions,  such  as  the  use  of  military  systems 
bought  with  little  modification  (sometimes  termed 
Military  Off  The  Shelf,  or  MOTS)  or  the  use  of  products 
developed  by  government  (Government  Off  The  Shelf, 
or  GOTS).  However,  MOTS  and  GOTS  items  share  a 
number  of  characteristics  with  COTS,  and  this  article 
may  offer  a number  of  insights  of  value  to  those  involved 
in  such  acquisitions. 

Basic  Characteristics  of  COTS  Products 

The  basic  characteristics  of  COTS  components  stem 
from  the  fact  that  they  are  developed  for  commercial, 
rather  than  military,  purposes  and  that  they  are  sold  in 
large  numbers  (sometimes  millions).  COTS  products 
have  been  designed  to  make  a profit  for  the  vendor,  and 
not  for  the  convenience  of  the  (minority)  military 
customer.  Upgrades  and  changes  are  driven  by  predicted 
return  on  investment  and  not  by  some  altruistic  desire  to 
improve  or  extend  a product.  The  military  user  generally 
represents  a small  minority  of  the  customers  of  a given 
COTS  product,  and  military  specific  features  are  unlikely 
to  appear  high  on  the  list  of  priorities  for  the  vendor. 

It  is  not  the  intention  of  this  paper  to  provide  a detailed 
description  of  the  advantages  and  disadvantages  of  using 
COTS  products.  However,  the  following  section 
summarise  the  main  points,  to  put  the  management 
problem  in  context. 

Advantages:  The  advantages  of  using  COTS  products 
have  been  advertised  widely  (possibly  too  widely).  They 
include  the  following: 

• Low  initial  cost,  with  development  costs  amortised 
over  many  buyers 

• Availability  of  established  support  arrangement, 
including  development  tools,  vendor  support  and 
spare  part  support 

• Reduced  acquisition  times  by  the  use  of  standard 
pre-developed  components 

• Ability  to  capitalise  on  upgrades  in  technology 
developed  for  the  commercial  market 

• Ability  to  adapt  to  meet  new  requirements 

• Potential  for  enhanced  interoperability 

Disadvantages 

The  use  of  COTS  products  is  not  all  good  news.  In 
particular,  COTS  products  suffer  from 


• Rapid  obsolescence,  with  support  and  spares 
lifetimes  driven  by  commercial  markets  beyond  the 
control  of  the  defence  sector 

• Lack  of  product  control  with  changes  being  made  to 
meet  commercial  drivers 

• Lack  of  Design  Detail  leading  to  difficulties  in 
modifications  and  in  safety  and  security 
certification. 

• Mismatch  with  Military  Standards 


COTS-BASED  SYSTEMS  AND  COMPLEXITY 

The  complexity  of  developing  a COTS  based  system  is 
often  underestimated.  It  is  important  to  recognise  that 
there  is  a considerable  difference  between  buying  a 
complete  COTS  system,  sold  commercially  in  the  form 
or  configuration  that  the  military  will  use,  and 
developing  a system  based  on  COTS  components 
(referred  to  as  "COTS-based  systems"  in  this  article). 
Lack  of  recognition  of  this  COTS  characteristic  has  been 
at  the  root  of  many  management  issues  in  the 
development  of  military  COTS  based  systems. 

There  has  been  an  impression  that  COTS-based  systems 
are  easy  to  build,  and  therefore  the  use  of  COTS  will 
automatically  reduce  design  complexity  and  hence  cost, 
timescales  and  risks.  This  feeling  has,  to  some  extent, 
been  generated  by  an  incorrect  extrapolation  from  the 
observed  characteristics  of  complete  COTS  system 
purchases.  When  a complete  COTS  system  is  purchased, 
the  system  design  has  been  carried  out  by  the  vendor  and 
its  complexity  is  hidden  from  the  purchaser.  Design  cost 
has  been  amortised  over  a large  number  of  purchasers, 
reinforcing  the  impression  that  the  cost  of  COTS-based 
system  design  is  low. 

Unfortunately,  this  assumption  is  not  valid  for  a typical 
military  COTS-based  systems,  which  will  contain  a large 
number  of  COTS  components  or  products,  each  of  which 
is  purchased  separately  from  the  vendor  and  then 
integrated  to  form  a new  system  configuration,  never 
previously  developed  and  unique  to  this  application. 
This  integration  will  involve  the  configuration  of 
individual  products  to  match  their  environment  and 
typically  require  the  development  of  custom  code  to 
provide  interfacing  functionality  and  to  meet  the  specific 
system  requirements. 

The  unique  configuration  will  also  place  the  individual 
components  in  a new  environment,  never  tried  before, 
and  this  may  well  expose  incompatibilities  previously 
unknown  to  either  the  developers  or  the  COTS  vendors. 
The  situation  is  further  complicated  in  most  military 
systems  by  the  need  for  bespoke  applications  to  meet 
specific  military  requirements  and  the  need  to 
incorporate  bespoke  legacy  applications. 

As  a consequence  of  these  issues,  COTS-based  systems 
require  at  least  as  much  effort  in  system  design  as  any 
system  based  on  bespoke  components.  Indeed,  it  may  be 
argued  that  the  fixed  functionality  and  performance  of 
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the  COTS  components  place  greater  constraints  on  the 
design  of  the  system,  forcing  more  iteration  between 
system  levels.  This  design  iteration  will  not  cease  when 
the  initial  design  is  completed. 

In  summary,  the  combination  of  a unique  design,  the  use 
of  a large  number  of  inflexible  components  in  a new 
environment  and  a mix  of  bespoke  and  COTS  elements, 
means  that,  contrary  to  widely  held  opinion,  large 
COTS-based  systems  are  inherently  complex. 
Management  plans  that  fail  to  recognise  this  complexity 
are  likely  to  underestimate  the  effort  and  time  required 
for  system  design,  both  during  initial  acquisition  and 
during  the  in  service  life  of  a system. 


CONTINUOUS  DESIGN  PROCESS 

As  the  underlying  COTS  components  are  replaced  by 
others  (as  they  surely  will  be),  the  system  configuration 
or  design  needs  revisiting  to  address  the  characteristics 
and  functionality  of  the  new  components.  In  some  cases 
the  changes  will  be  minor,  for  instance  when  a 
component  is  superseded  by  another  without  affecting  its 
functionality  or  interfacing,  and  the  effort  required  will 
principally  be  focussed  on  configuration  management. 
In  other  cases,  however,  the  withdrawal  of  support  for  a 
key  infrastructure  component  (such  as  an  operating 
system  or  database)  may  necessitate  a major  redesign 
with  impact  on  many  other  components  in  the  system. 

The  interrelated  nature  of  IT  products  can  lead  to  a 
domino  effect,  with  the  change  of  one  component 
requiring  the  replacement  of  many  others.  For  example 
the  change  to  a new  processor  could  require  a new 
operating  system,  which  may  in  turn  require  application 
programmes  to  be  replaced.  It  may  also  require  the 
redesign  of  bespoke  application  software  developed  for 
the  system.  The  unique  nature  of  a given  military  system 
also  means  that  with  each  change,  components  may  be 
placed  in  a new  environment,  which  can  expose 
shortcomings  in  products  not  previously  uncovered. 
This  will  need  to  be  resolved  before  the  system  is  put 
into  service.  Each  major  increment  will,  of  course,  also 
bring  the  need  for  extensive  testing  and  revalidation. 

The  rapid  turnover  of  COTS  products  and  the 
consequent  changes  to  the  system  design  and 
configuration  means  that  a COTS-based  system  is  in  a 
state  of  continuous  design,  throughout  its  lifetime.  (See 
Figure  1) 

This  fact  needs  to  be  recognised  in  the  through  life 
management  of  a COTS-based  system,  and  suitable 
resources  and  funding  to  support  the  continuous  design 
process  must  be  secured. 


MAGNITUDE  OF  TECHNOLOGY  CHANGES 

It  is  readily  apparent  that  the  technology  on  which  COTS 
products  are  based  will  change  during  the  lifetime  of  a 
typical  military  system.  While  it  is  universally 


recognised  that  that  changes  will  take  place  in 
technology,  discussions  with  a wide  range  of  military 
projects  suggests  that  the  magnitude  of  the  changes  is 
often  not  appreciated  or  taken  into  account  in  project 
management  planning. 

To  get  some  idea  of  the  likely  impact  of  technology 
changes  on  military  systems  we  need  to  look  forward 
some  twenty  five  years  (at  the  end  of  which  many 
systems  currently  in  the  concept  stage  will  still  be  in 
service).  If  we  look  back  twenty-five  years,  to  1975,  we 
can  see  how  far  commercial  information  technology  has 
moved.  In  1975,  there  were  no  desktop  computers,  no 
Internet  (in  the  form  we  would  recognise  today)  and  no 
mobile  phones.  Object  oriented  programming  was  an 
obscure  specialist  technique  and  interfaces  were  (at  best) 
text  based.  The  microprocessor  was  in  its  infancy  (the  6 
MHz  8080  and  6.4  MHz  6800  were  both  launched  in 
1974).  Windowed  user  interfaces,  mice,  LCD  screens, 
the  world  wide  web,  TCP/IP  and  HTML  were  still  all 
years  in  the  future.  Figure  2 shows  some  of  the  key 
events  over  the  last  twenty-five  years.  In  short,  we  can 
see  that  commercial  technology  has  changed  beyond  all 
recognition. 

It  is  generally  considered  that  the  rate  of  change  of 
technology  has  been  increasing  over  this  period,  and 
today  new  concepts  and  ideas  are  being  introduced  at  a 
high  rate.  (The  life  of  a commercial  software  product  is 
typically  12-18  months  before  it  is  replaced  by  a new 
version,  and  some  2-3  years  before  all  support  is 
dropped.)  It  is  against  this  background  that  we  are 
asking  industry  to  develop  systems  that  will  last  for 
twenty  or  more  years  beyond  In  Service  Date  (ISD). 

Some  changes  during  this  time  will  be  predictable.  The 
cost  of  processing  power  will  continue  to  fall,  bandwidth 
available  to  commercial  users  will  expand,  and  the  cost 
of  storage  (volatile  and  non-volatile)  will  reduce. 
However,  as  the  last  ten  or  twenty  years  has  shown  us, 
the  way  in  which  these  developments  will  be  exploited  in 
the  commercial  world  is  impossible  to  predict. 

If  specific  technology  trends  can't  be  predicted,  those 
considering  the  design  and  implementation  of  COTS- 
based  systems  must  consider  the  magnitude  of  the 
changes  that  are  likely  to  take  place.  In  the  next  15  years 
we  will  see  changes  as  far  reaching  as:  the  removal  of 
keyboards  and  screens  as  interface  devices,  the  demise  of 
a web  based  approach  or  indeed  of  the  internet  as  we 
know  it,  the  advent  of  effectively  unlimited  bandwidth 
for  commercial  users  (with  the  subsequent 
transformation  in  commercial  system  architectures  and 
techniques)  or  the  demise  of  the  concept  of  a workstation 
running  software.  It  is  not  suggested  that  all  (or  any)  of 
these  specific  possibilities  will  definitely  occur,  but 
changes  of  this  magnitude  are  certain  to  arise.  The 
challenge  to  military  COTS-based  systems  designers  is 
to  develop  architectures  and  design  and  management 
approaches  that  can  deal  with  this  level  of  innovation 
during  the  25-year  life  of  a typical  military  project. 
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The  rate  of  change  of  technology  means  that  a system 
will  have  to  deal  with  more  than  just  component 
obsolescence  during  its  lifetime.  In  the  typical  25-year 
life  of  a system,  commercial  technology  may  be  expected 
to  have  changed  beyond  recognition.  Current  standards, 
design  approaches  and  architectures  will  have  been 
superseded  and  forgotten.  There  is  very  little  scope  for 
assuming  that  we  could  continue  to  use  today's  hardware 
and  software  solutions  throughout  the  life  of  a military 
COTS  based  system.  Even  though  we  do  not  know 
exactly  what  the  changes  will  be,  we  must  plan  to 
manage  this  level  of  technology  change  if  fatal 
obsolescence  problems  are  to  be  avoided. 


REQUIREMENTS  MANAGEMENT 

All  studies  into  the  use  of  COTS  in  military  systems 
emphasise  the  need  for  a suitable  process  to  manage 
requirements  and  requirement  trade-offs.  It  is 
considered,  however,  that  we  have  yet  to  see  a system  or 
management  approach  that  handles  this  task 
satisfactorily. 

If  a COTS  product  is  to  be  used  in  a system,  there  is  very 
little  scope  to  change  its  functionality  (although  many 
products  have  parameters  and  settings  that  can  be 
changed).  When  a COTS  product  is  selected,  it  is  highly 
unlikely  that  its  characteristics  will  match  precisely  those 
of  the  requirement.  This  implies  that  it  may  be  sensible 
to  accept  the  capability  offered  by  the  product  despite  the 
fact  that  it  is  not  precisely  what  was  originally  demanded 
by  the  user. 

As  the  design  becomes  more  detailed,  and  different 
combinations  of  products  are  selected,  then  the  match  of 
these  to  the  original  specification  will  need  to  be 
assessed.  In  some  cases  the  advantages  (low  price, 
availability,  good  support)  offered  by  a COTS  product 
will  outweigh  the  fact  that  it  does  not  match  the  original 
requirement.  In  other  cases,  there  may  be  a need  to 
select  a different  product,  or  use  the  product  and  enhance 
its  capability  by  the  use  of  other  products,  or  by 
producing  some  bespoke  application  code  to  provide  the 
required  functionality.  In  many  cases,  of  course,  a 
COTS  product  will  have  features  that  were  not  originally 
included  in  the  requirement,  but  which  are  of  value  to  the 
customer.  Design  decisions  such  as  these  can  only  be 
carried  out  if  the  design  team  has  the  skills  and 
experience  to  understand  the  needs  of  the  user,  and  the 
impact  of  any  possible  design  changes.  This  will  require 
a very  close  relationship  between  the  designer/system 
integrator  and  the  user  community. 

Trade-offs  between  cost,  risk,  availability  and 
functionality  will  continue  throughout  the  life  of  the 
system.  As  obsolescence  forces  the  change  to  a 
component,  the  selection  of  its  replacement  will  require 
the  same  assessments  to  be  carried  out,  possibly  leading 
to  further  agreed  changes  to  the  user's 


expectations/requirement.  The  timescales  of  changes 
will  often  mean  that  later  fielded  systems  are  different 
from  their  predecessors,  leading  to  the  potential  for  a 
range  of  different  systems  in  service,  each  developed 
during  trade-offs  for  particular  systems  or  batches  of 
systems. 

An  additional  feature  of  COTS-based  systems  is  that  the 
use  of  commercial  technology  should  allow  the  rapid 
exploitation  of  advances  in  the  commercial  world.  This, 
in  turn,  means  that  COTS-based  systems  offer  the  chance 
to  enhance  the  requirement  in  a cost-effective  manner. 
In  particular  it  should  allow  military  systems  to  exploit 
new  applications  and  methods  of  working  in  the 
commercial  world.  The  management  of  upgrades  will 
need  careful  control;  this  is  discussed  further  below. 

The  aspects  discussed  above  indicate  that  if  the  full 
potential  of  COTS-based  systems  are  to  be  exploited, 
then  a close  and  dynamic  relationship  is  required 
between  end  users,  procurement  staff  and  industry.  This 
close  working  relationship  will  be  required  throughout 
the  life  of  the  system,  as  trade-offs  and  requirement 
developments  are  initiated  by  COTS  product  changes 
forced  by  obsolescence  and  upgrades.  Such 
relationships  are  rare,  with  traditional  acquisition 
approaches  often  leading  to  a confrontational 
relationship,  rather  than  close  cooperation. 

A key  to  making  sound  decisions  in  this  dynamic 
environment  will  be  a clear  understanding  by  all  parties 
of  the  way  in  which  commercial  technology  is 
advancing.  Such  knowledge  will  permit  a realistic  vision 
of  what  is  likely  to  become  feasible  in  the  near  future 
and  will  assist  in  foreseeing  and  managing  potential 
obsolescence  problems. 

In  summary,  COTS-based  systems  involve  considerable 
effort  to  be  placed  on  requirement  negotiation  and  trade- 
off. This  process  needs  to  involve  all  stakeholders,  and 
many  of  the  detailed  decisions  will  continue  to  be 
required  beyond  Main  Gate.  Requirements  evolution  will 
continue  throughout  the  life  of  the  system,  as  new 
products  are  delivered  by  developments  in  technology 
and  old  products  become  unsupportable. 

The  use  of  rapidly  developing  commercial  components 
brings  the  need  for  a paradigm  shift  in  requirements 
management.  Conventional  tools  and  methods  are 
inadequate  to  either  capitalise  on  the  huge  advantages 
that  COTS  products  could  deliver  or  to  avoid  the  pitfalls 
of  obsolescence.  A much  greater  emphasis  is  required 
on  involving  all  stakeholders  and  a new  approach  to 
requirements  management  is  required,  based  on 
continuous  requirements  evolution.  If  the  full  potential  of 
COTS-based  systems  is  to  be  exploited,  then  a close  and 
dynamic  relationship  is  required  between  users, 
procurement  authorities  and  industry. 
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COST  FORECASTING  AND  FINANCIAL 
MANAGEMENT 

Successful  project  management  includes  a need  to 
predict  future  costs,  and  plan  future  spending  and 
manage  the  programme  to  remain  within  allocated 
budgets.  In  the  procurement  of  military  systems,  it  has 
long  been  recognised  that  initial  acquisition  costs  are 
heavily  outweighed  by  the  costs  of  in  service  support, 
and  this  has  recently  led  to  an  emphasis  on  through  life 
or  whole  life  costs. 

Unfortunately,  however,  accurate  prediction  of  the  long- 
term costs  of  a complex  COTS-hased  system  is  not 
possible,  for  a number  of  reasons.  These  include: 

• A lack  of  suitable  cost  models. 

• No  agreed  MOD/industry  process  for  the 
maintenance  and  development  of  COTS-based 
systems  through  life,  making  assessment  of  through 
life  costs  infeasible. 

• Volatility  and  unpredictability  of  future 
developments  in  technology. 

• Unpredictability  of  the  direction  of  future 
commercial  developments  and  their  applicability  to 
military  systems. 

This  poses  particular  problems  in  military  system 
acquisition,  in  which  acquisition  authorities  are  expected 
to  provide  reliable  through  life  cost  estimates  early  in  the 
programme  to  support  the  choice  of  contractor  or 
solution.  There  may  be  reasonable  assessments  of  costs 
through  the  initial  acquisition  of  the  system  but  cost 
estimates  beyond  this  will  be  subject  to  significant 
uncertainty. 

It  is  not  possible  to  accurately  predict  the  through  life 
cost  of  a COTS  based  system.  This  fact  needs  to  be 
recognised  in  the  planning  and  funding  of  systems,  and 
runs  counter  to  most  standard  defence  acquisition 
strategies.  Failure  to  plan  for  this  uncertainty,  and  to 
secure  adequate  flexibility  in  funding  will  delay  the 
introduction  of  updates,  leading  to  increased  problems 
with  obsolescence  and  support. 


SAFETY  AND  SECURITY  MANAGEMENT 

The  use  of  COTS  products  in  systems  introduces  some 
significant  difficulties  with  regard  to  system  integrity 
assessments,  in  particular  for  safety  and  security 
assessment  and  accreditation.  Those  problems  are 
caused  by  a number  of  factors,  including  the  way  that 
COTS  products  are  developed  and  controlled,  the  lack  of 
information  and  the  rapid  obsolescence  and  replacement 
of  products. 

The  military  world  has  traditionally  insisted  on  systems 
meeting  specific  standards  regarding  product  safety.  It 
can  generally  be  assumed  that  COTS  products  will  not 
have  been  designed  to  meet  these  specific  standards  , 
although  in  some  cases  equivalent  civil  standards  will 
have  been  addressed.  In  some  cases  these  standards  will 
be  acceptable  for  use  in  the  military  environment,  and 


the  explicit  requirement  to  meet  a military  standard  can 
be  waived.  If  this  is  not  the  case,  however,  the  military 
system  procurer  may  have  to  gather  further  evidence  or 
undertake  specific  tests  on  the  proposed  or  delivered 
products  to  assess  their  safety.  Such  tests  may  not, 
however,  be  cheap  and  gaining  assurance  that  they  will 
remain  valid  for  all  deliveries  may  be  difficult.  For 
example  the  source  and  chemical  make-up  of  cases  and 
components  may  change  between  batches,  and  toxicity 
tests  undertaken  on  a sample  product  may  not  be 
representative  of  all  such  products.  If  assurances  are 
required  that  tests  will  remain  valid,  then  a manufacturer 
may  have  to  establish  additional  procedures  or  a different 
product  line,  in  each  case  this  will  invite  additional  costs. 

COTS  software  presents  particular  difficulty  in  safety 
related  (or  safety  critical)  systems,  because  lack  of 
control  over  the  development  method  and  lack  of 
information  render  standard  methods  of  assessing 
software  quality  infeasible.  In  particular: 

• COTS  products  have  already  been  designed,  and  so 
design  and  coding  methods  (such  as  the  use  of 
formal  methods)  can  not  be  influenced. 

• Code  listings  are  not  generally  available  for  COTS 
software  products,  rendering  static  code  analysis 
impossible. 

• COTS  products  will  generally  have  been  designed  to 
less  rigid  standards  than  those  demanded  by,  for 
instance,  Def  Stan  00-55. 

• Large  software  infrastructure  components  (such  as 
operating  systems)  are  of  a complexity  that  renders 
exhaustive  testing  impossible. 

Even  if  a product  had  been  analysed  and  accepted,  the 
short  lifetime  of  COTS  products  can  force  repeated 
analysis.  Any  analysis  will  take  time,  and  this  may 
introduce  delays  in  re-confirming  the  safety  or  security 
accreditation  of  the  system. 

These  difficulties  can  be  mitigated  by  good  system 
design  (for  instance  by  partitioning  of  safety  critical 
elements  of  a system,  or  by  adding  additional  safety 
controls),  and  by  using  alternative  assessment  methods 
(for  example  assessing  the  general  quality  of  a 
company's  software  or  gathering  evidence  on  the 
reliability  of  the  COTS  software  product).  The  selection 
of  the  supplier  should  include  an  analysis  of  his 
credentials  and  qualifications  in  supplying  safety  critical 
software.  The  assessment  of  the  system  and  the 
accreditation  task  itself  will  be  simpler  if  the  supplier  has 
a suitable  track  record  and  is  familiar  with  the 
development  and  accreditation  of  safety  critical  items. 

The  safety  and  security  accreditation  of  COTS  based 
systems  presents  considerable  technical,  design  and 
management  challenges.  The  difficulties  and  cost  of 
initial  and  ongoing  system  accreditation  must  be 
considered  in  the  development  of  COTS-based  systems 
and  in  the  selection  of  system  contractors.  Delays  in  the 
introduction  of  upgrades  caused  by  safety  and  security 
issues  will  increase  obsolescence  problems. 
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MANAGEMENT  OF  SYSTEM  UPGRADES 

The  key  to  successful  ongoing  support  and  improvement 
of  a COTS-based  system  will  be  the  development  of  a 
suitable  infrastructure,  into  which  new  components  and 
products  can  be  inserted,  combined  with  the 
development  of  a suitable  management  regime.  The 
implementation  of  an  open,  flexible  infrastructure, 
capable  of  adaptation,  extension  and  scaling  to  counter 
obsolescence  and  to  provide  new  functionality  and 
capacity,  is  not  a simple  task.  Those  financing  and 
approving  programmes  will  have  to  take  into  account 
that  it  is  more  expensive  to  develop,  implement  and 
maintain  such  an  infrastructure  than  to  develop  one  that 
will  simply  meet  the  current  demands. 

Management  of  upgrades 

The  terminology  relating  to  system  modifications  and 
upgrades  is  complex,  diverse  and  inconsistent.  It  is 
important  to  distinguish  between  at  least  two  different 
categories  of  system  modification.  These  are: 

• Changes  driven  by  obsolescence  ("technology 
refresh") 

• Changes  to  increase  capability  ("capability 

upgrades") 

Having  made  that  distinction,  however,  we  must 
recognise  that  there  are  limited  opportunities  for 
upgrades,  and  the  need  for  cost  effectiveness  means  that 
any  significant  system  upgrade  event  will  include 
elements  from  both  of  these  categories.  As  the  different 
categories  of  modification  carry  different  responsibilities 
for  specification  and  funding,  this  has  the  potential  to 
introduce  management  difficulties. 

The  management  of  upgrades  in  a COTS-based  system  is 
a far  from  simple  problem.  It  involves  a wide  range  of 
stakeholders  with  conflicting  interests,  and  successful 
resolution  will  require  understanding  of  many 
viewpoints  and  interests.  There  are  many  tightly 
interrelated  factors  to  be  considered  in  managing  system 
development  and  in  planning  individual  upgrade  events. 
These  include  cost,  time  required  to  implement  the 
upgrade,  time  required  for  preparation,  risk, 

obsolescence  pressures,  availability  of  COTS  and  legacy 
components,  platform  programmes,  links  with  other 
programmes  and  specific  operational  demands.  (Figure 
3).  These  various  aspects  will  need  to  be  assessed  and 
traded  off  in  any  particular  upgrade,  and  this  will  require 
input  and  understanding  by  all  stakeholders.  The 
complex  interrelationship  between  the  various 

stakeholders  will  be  simplified  by  clear  understanding  of 
their  individual  aims  and  responsibilities.  Managing  the 
upgrade  process  will  require  the  co-operation  and 
support  of  all  stakeholders. 

A through  life  view  needs  to  be  taken  by  all  parties,  and 
each  must  have  an  incentive  to  act  in  a manner  consistent 
with  getting  overall  value  for  money  on  a through  life 
basis.  Amongst  other  things,  this  means  that: 


• Those  controlling  the  finance  must  recognise  that 
there  will  often  be  an  up-front  cost  to  keep  a system 
flexible  enough  to  accommodate  future  (but 
currently  unknown)  capability  upgrades. 

• The  acquisition  authority  must  recognise  that 
industry  needs  to  make  a profit,  and  enter  into 
arrangements  that  allow  for  this  while  still  ensuring 
good  value  for  money. 

• Industry  must  be  given  the  incentive  to  invest  in 
system  upgrades  and  support  facilities  confident  in 
the  belief  that  these  will  contribute  to  increased 
return  at  a later  date. 

• The  long  term  strategy  for  maintaining  and 
upgrading  the  system  needs  to  be  agreed  early  in  the 
programme,  in  order  that  the  through  life  costs  can 
be  realistically  estimated  and  suitable  support 
arrangements  put  in  place. 

Management  plans  must  recognise  the  complexities  of 
managing  technology’  refresh  and  capability  upgrades. 
Successful  management  of  upgrades  will  require  the 
cooperation  of  many  stakeholders,  often  with  different 
and  conflicting  priorities.  Successful  management  of 
upgrades  will  only  be  possible  if  there  is  a close  and 
trusted  working  relationship  between  MOD  and  industry. 
The  confrontational  approach  that  is  typical  of  many 
current  procurements  will  preclude  cost  effective 
management. 


CONTRACTOR  LOGISTIC  SUPPORT 

Contractor  Logistic  Support  (CLS)  contracts,  where  the 
contractor  is  given  the  responsibility  for  supporting  a 
system  for  a given  period,  are  often  seen  as  a standard 
solution  for  reducing  risk  on  acquisition  authorities  and 
gaining  cost  effective  support  for  a system.  However, 
for  COTS-based  systems,  there  is  a danger  that,  unless 
supported  by  other  incentive  schemes,  the  traditional 
CLS  contract  can  contribute  to  increased  obsolescence 
and  higher  through  life  costs. 

As  has  been  pointed  out,  COTS-based  systems  suffer 
from  rapid  obsolescence,  leading  to  the  need  for 
continuous  technology  refresh  if  they  are  to  remain 
supportable.  If  a contractor  accepts  a firm  price  CLS 
contract  for,  say,  the  five  years  following  ISD  then  he 
will  be  under  an  obligation  to  support  the  system,  and 
hence  it  will  be  in  his  interests  to  keep  the  system  free 
from  obsolescence  problems  during  this  period. 
However,  he  will  not  wish  to  spend  more  money  than  is 
necessary  to  meet  his  contractual  commitments.  As  the 
end  of  the  CLS  period  approaches,  the  system  will  be  in 
a state  that  no  further  technology  refresh  is  required  to 
maintain  the  system  for  the  remainder  of  the  period. 
This  point  will  probably  be  some  three  years  before  the 
end  of  the  period.  The  contractor  might  not,  therefore, 
undertake  any  work  to  mitigate  against  future 
obsolescence  during  these  three  years.  The  result  will  be 
that  at  the  end  of  the  CLS  period,  the  system  will  be 
about  to  become  unsupportable. 
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To  avoid  this,  there  is  a need  to  provide  the  contractor 
with  the  incentive  to  keep  the  through  life  cost  of 
obsolescence  low.  A fixed  CLS  period,  with  no  further 
obligation  or  commitment,  will  only  provide  an  incentive 
to  keep  the  obsolescence  cost  low  during  the  contracted 
period.  It  is  clear  that  we  will  have  to  look  at  innovative 
solutions  to  this  problems.  These  will  involve  working 
closely  with  their  suppliers  to  achieve  solutions  that  are 
of  mutual  benefit.  For  these  solutions  to  be  successful 
through  life,  the  benefits  will  have  to  be  capable  of  being 
shared  between  government  and  industry. 

The  management  of  CLS  for  a COTS-based  system 
requires  careful  consideration,  as  the  conventional 
"hands  off  approach  brings  particular  problems.  A 
closer  working  relationship,  and  cost  and  risk  sharing, 
will  be  required  if  a successful  support  regime  is  to  be 
maintained. 


SPARES  SUPPORT  AND  CONFIGURATION 
CONTROL 

The  use  of  COTS  components  in  a complex  system 
introduces  some  significant  difficulties  in  the  domain  of 
configuration  management  and  spares  support.  These 
challenges  are  a direct  result  of  the  fundamental 
characteristics  of  COTS  products,  including: 

• short  periods  of  commercial  availability, 

• interdependence  between  products  (including 
hardware  and  software  interdependencies) 

• the  potential  supply  of  compatible  COTS 
components  from  a number  of  suppliers. 

The  principles  of  configuration  management  are  as  (or 
more)  important  in  COTS-based  systems  as  they  are  in 
traditional  systems.  However  the  widespread  use  of 
COTS  products  introduces  a number  of  additional 
complexities  to  configuration  management.  These 
include: 

• frequent  design  changes, 

• lack  of  configuration  information  for  COTS 
products, 

• inter-dependence  between  hardware  and  software, 

• need  to  track  the  installation  of  new  versions  even 
when  they  appear  to  be  completely  interchangeable, 

• the  many  minor  changes  made  to  new  COTS 
software  products, 

• the  lack  of  reliability  data. 

The  short  supply  lifetime  of  COTS  components  and  the 
diversity  of  configurations  make  the  supply  of  hardware 
spares  difficult  to  manage.  As  new  products  appear,  and 
old  versions  are  no  longer  available,  there  will  be  a 
requirement  to  certify  new  products  for  use  in  a system 
and  manage  their  supply  and  availability  for  the  different 
system  configurations  in  use. 

The  use  of  COTS  components  brings  the  potential  for  a 
configuration  explosion,  with  each  installation  (and  each 
sub-system  within  the  installation)  being  significantly 


different  from  others.  This  in  turn  brings  complexities 
for  spares  and  support  management.  A balance  will  have 
to  be  struck  between  containing  this  diversity  and  the 
cost  of  limiting  implementations  to  a manageable  subset 
of  configurations. 

Whole  life  buys  (or  "Through  life  buys"  or  "Lifetime 
buys")  are  often  proposed  as  a strategy  for  dealing  with 
hardware  obsolescence.  Unfortunately,  experience  has 
shown  that  these  are  rarely  a realistic  solution,  for  a 
number  of  reasons: 

• Inter-relationships  between  software  and  hardware  - 

COTS-based  systems  often  exhibit  a strong 
interdependence  between  their  components,  and 
particularly  between  the  software  (both 

infrastructure  and  application)  and  the  hardware  on 
which  it  runs.  In  the  commercial  world,  new 
processor  upgrades  are  commonplace,  and  as  new 
software  is  developed,  support  for  older  hardware  is 
often  dropped.  The  consequence  of  this  is  that  if 
hardware  is  not  upgraded  then  in  a relatively  short 
timescale,  software  cannot  be  upgraded  further  with 
a direct  effect  on  the  capability  of  the  system  to 
react  to  new  threats  and  requirements. 

• Loss  of  ability  to  exploit  new  technology  - If  the 
infrastructure  hardware  and  software  becomes 
frozen,  then  the  capacity  to  modify  the  system  to 
add  new  functionality  is  reduced.  Software 
packages  that  the  users  may  like  incorporated  into 
the  system  will  not  be  available,  because  a modern 
commercial  package  will  expect  and  require  up  to 
date  or  recent  versions  of  the  operating  system, 
processors,  peripherals  etc. 

• Need  for  system  development  support  environment  - 
In  the  longer  term  the  decision  to  limit  the  system  to 
obsolete  technology  will  affect  the  development 
environment  as  well  as  the  system  itself.  For 
example,  as  new  software  languages  are  developed, 
compilers  will  only  be  written  for  newer  processors 
and  operating  systems.  This  will  further  limit  the 
ability  to  upgrade  the  system. 

• Difficulty  in  predicting  numbers  - It  is  difficult  to 
gather  or  obtain  MTBF  figures  for  COTS  products, 
either  because  the  data  has  not  been  gathered,  or 
because  they  have  relatively  short  life  histories,  or 
because  they  have  not  been  used  in  representative 
environments.  This  lack  of  data,  combined  with  the 
possibility  of  the  spares  being  rendered  unsuitable 
by  other  changes  in  the  system,  represents  a major 
risk  in  costing  and  undertaking  whole  life  buys  of 
spares. 

Spares  support  for  COTS-based  system  presents  a 
complex  management  challenge,  with  the  potential  for 
serious  configuration  management  problems.  The 
principles  of  configuration  management  are  just  as 
important  for  a COTS-based  system  as  for  any  other 
procurement,  but  the  use  of  COTS  products  brings  the 
potential  for  an  explosion  in  system  and  subsystem 
configurations.  This  diversity  carries  a cost  overhead, 
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and  will  need  to  be  contained.  It  is  essential  that  this 
issues  is  addressed  in  other  management  areas, 
including  technical  design,  funding  and  support 


management.  The  use  of  through  life  buys  of  spares  is 
rarely  an  adequate  solution  to  these  problems. 


CONCLUSIONS 

Military  acquisition  is  moving  into  new  and  uncharted  waters.  The  use  of  COTS  products  as  the  basis  for  military 
systems  brings  many  advantages,  but  it  also  brings  many  challenges.  To  meet  these  challenges  will  certainly  require 
new  technical  skills  and  an  understanding  of  the  characteristics  of  COTS  within  procurement  organisations.  However, 
COTS-based  systems  also  bring  management  challenges,  many  of  which  run  counter  to  current  working  practices  in 
military  acquisition.  If  we  are  to  rise  to  these  challenges,  and  harness  the  potential  advantages  of  COTS,  then  a change 
in  management  culture  and  philosophy  will  be  required,  allowing  the  introduction  of  new  management  approaches, 
representing  and  balancing  the  needs  of  all  stakeholders  in  government  and  industry. 
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Figure  1 - Comparison  of  traditional  and  COTS-based  system  acquisition  lifecycles 
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Figure  2 - Selected  events  in  the  development  of  commercial  technology 
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